Method for estimating statistics of properties of memory system transactions

ABSTRACT

A method estimates statistics of properties of transactions processed by a memory sub-system of a computer system. The method randomly selects memory transactions processed by the memory sub-system. States of the system are recorded as samples while the selected transaction are processed by the memory sub-system. The recorded states from a subset of the selected transactions are statistically analyzed to estimate statistics of the memory transactions.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is related to: U.S. Pat. Applications Ser. NOs.08/980,170, which is now pending, and 08/980,167, 08/980,105,08/977,438,08/980,124; 08/980,189, 08/979,033, 08/979,034, 08/980,168,08/979,822, 08/979,398,08/979,899 now U.S. Pat. No. 5,809,450,08/980,190 now U.S. Pat. No, 6,000,044, 08/980 166 now U.S. Pat. No.6,070,009, 08/980,145, now U.S. Pat. No. 5,964,867 08/980,035 now U.S.Pat. No. 6,092,180, 08/08/980,164 now U.S. Pat. No. 6,119,075, and08/979,848 now U.S. Pat. No. 5,923,872; all of which are assigned to theassignee of the present application.

FIELD OF THE INVENTION

The present invention relates generally to measuring the performance ofcomputer systems, and more particularly to estimating statistics ofmemory sub-system interactions.

BACKGROUND OF THE INVENTION

The speed at which modem computer systems operate is often limited bythe performance of their memory sub-systems, such as caches and otherlevels of a hierarchical memory subsystem containing SRAM, DRAM, disksand the like. Cache memories are intended to store data that sharespatial and temporal localities. Other memories can store data in anynumber of organized manners, short term and long, term.

In order to analyze and optimize the performance of memory transactions,better measuring tools are required. Currently, there are very few toolsthat can accurately measure and capture detailed informationcharacterizing memory transactions.

Existing hardware event counters can detect discrete events related tospecific memory transactions, such as cache references, or cache misses,but known event counters provide little detail that would allow one toexactly deduce the causes of performance debilitating events, and howsuch events could be avoided.

For example, currently it is extremely difficult to obtain informationabout the status of a cache block, such as clean or dirty, or shared ornon-shared, while data are accessed. It is also very difficult todetermine which memory addresses are actually resident in the cache, orwhich memory addresses are conflicting for a particular cache block,because existing systems do not provide an easy way to obtain thevirtual and physical address of the data that are accessed.

Similarly, it is difficult to ascertain the source of a particularmemory reference that caused a performance debilitating event. Thesource might be an instruction executed in the processor pipeline onbehalf of a particular context (e.g., process, thread, hardware context,and/or address space number), it might be a memory request that isexternal to the processor pipeline, such as direct memory access (DMA)originating from various input/output devices, or it may be acache-coherency message originating from other processors in amultiprocessor computer system. Sampling accesses to specific regions ofmemories, such as specific blocks in lines of a cache, physicaladdresses in a main memory, or page addresses in a virtual memory iseven more difficult.

It may be possible, using simulation or instrumentation, to track memoryaddresses for processor initiated accesses, such as those due to loadand store instructions. However, simulation and instrumentationtechniques usually disturb the true operation of the system enough togive less than optimal measurements, particularly for large scalesystems with real production workloads. Also, because instrumentationtechniques modify or augment programs, they inherently alter memory andcache layouts, distorting the memory performance of the original system.For example, instruction cache conflicts may differ significantlybetween instrumented and uninstrumented versions of a program.

However, when the memory accesses are due to some event, such as a DMAtransaction or a cache coherency transaction in a multi-processor,tracking accessed addresses can usually only be done by specializedhardware designed specifically for that part of the memory subsystemwhich is to be monitored.

In addition, in order to optimize operating system and applicationsoftware, it would be useful to be able to obtain other types ofinformation about memory transactions, such as the amount of memory thatis used by different execution threads or processes, and the amount oftime required to complete a particular memory transaction. Furthermore,it would be even more useful if the information could be used tooptimize instruction scheduling and data allocation, perhaps even whilethe system is operating under a real workload.

SUMMARY OF THE INVENTION

Provided is a method for estimating statistics of properties oftransactions processed by a memory sub-system of a computer system. Themethod randomly selects memory transactions processed by the memorysub-system.

States of the system are recorded as samples while the selectedtransaction are processed by the memory sub-system. The recorded statesfrom a subset of the selected transactions are statistically analyzed toestimate statistics of the memory transactions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a computer system with memory transactionsampling according to the invention;

FIG. 2 is a block diagram of sampling hardware for monitoring memoryperformance according to a preferred embodiment;

FIG. 3 is a block diagram of a sampling buffer to store sampleinformation;

FIG. 4 is a flow diagram of a method for estimating sharing and conflictstatistics about memory system interactions between computer systemcontexts;

FIG. 5 is a flow diagram of a method for estimating statistics ofproperties of memory system transactions;

FIG. 6 is a flow diagram of a method for using statistics about memorysystem behavior to make data replication and migration decisions; and

FIG. 7 is a flow diagram of a method for using statistics about memorysystem interactions to make context scheduling decisions.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT System Overview

FIG. 1 shows a computer system 100 which can use the memory transactionsampling techniques as described herein. The system 100 includes one ormore processors 110, memories 120, and input/output interfaces (I/O) 130connected by bus lines 140.

Each processor 110 can be implemented on an integrated semi-conductorchip including a processor pipeline 111, a data-cache (D-cache) 113, andan instruction cache (I-cache) 112, for example, the Digital EquipmentCorporation Alpha 21264 processor. The pipeline 111 can include aplurality of serially arranged stages for processing instructions, suchas a fetch unit, a map unit, an issue unit, one or more execution units,and a retire unit. The processor chip also includes hardware 119described in greater detail below for sampling cache state informationwhile accessing data stored at addresses in the various memories.

The memories 120 can be arranged hierarchically, including a board-levelcache (B-cache) 121, volatile memories (DRAM) 122, and persistentmemories (disk) 123. The I/O 130 can be used to input and output data toand from the system 100 using I/O devices such as memory channels toother processors, keyboards, monitors, and network controllers to othercomputer systems.

Memory Transaction

In general, a memory transaction is defined herein as any operationwhich causes data to move from one location to another, for example,loads and stores, direct memory access (DMA) operations, and coherencytransactions in the case where multiple processors or execution threadsaccess data concurrently.

Operation

During operation of the system 100, instructions and data of softwareprograms are stored in the memories 120. The instructions are generatedconventionally using known compiler, linker, and loader techniques. Theinstructions are transferred into the pipeline 111 of one of theprocessors 110 via the I-cache 112, and the data via the D-cache 113. Inthe pipeline 111, the instructions are decoded for execution.

The instruction cache (I-cache) 112 is accessed by the pipeline to fetchinstructions that are executed by the processor pipeline 111. Some ofthe instructions, for example load and store instructions, access data(R/W) stored in the memories via the D-cache 113. Other instructions,such as branch and jump instructions, control the execution flow of theprograms. Data can also be transferred via direct memory access (DMA)and cache coherency operations. It is desired to collect detailedperformance information while data in any of the memories are accessed.

Memory Transaction Sampling

FIG. 2 shows an arrangement for sampling memory transactions. A cache200 includes a plurality of lines 202. The cache can be direct-mapped,or set-associative. Each line consists of one or more blocks 201. Linesin a direct mapped cache will contain only a single block, while linesin an N-way set-associative cache will each contain N blocks.

For example, in a four-way set-associative cache, one line will storedata of four different memory addresses that have some number oflow-order address bits in common. During an access, after the line hasbeen referenced by the cache line index, each block has to be examinedto determine if the line stores the required data. This is done with atag 210. The exact details of how the blocks are examined depends on theimplementation of the set-associative cache. Also, associated with eachblock are the data 220, and status information 230. Different physicalhardware can be used for storing the tags, data, and status.

The arrangement shown in FIG. 2 also includes a translation-lookasidebuffet (TLB) 240, a trigger function 250, a counter 265, a selectionfunction 260, sampling buffers 301-303, and sampling software 260.

During operation of the system, transaction input 241 is presented tothe translation-lookaside buffer (TLB) 240 on line 241. The transactioninput can include a virtual address (VA), a context identifier such asan address space number (ASN), and in the case of a multi-threadedprocessor design, a hardware context identifier (HCI). The input alsocan include the type of the access operation to be performed (R/W/DMA).

The TLB 240 translates the virtual address to a physical address. Aportion of the address (the physical address for a physically-indexedcache, or the virtual address for a virtually-indexed cache) typicallyconsisting of some number of low-order (e.g. 8-16) bits, are used toform an index into the cache 200 on line 242′. The index selects aparticular cache line 202.

A lookup operation is then performed on each of the blocks 201 withinthe selected cache line 202 to determine if the appropriate data areresident in the blocks of the line. Depending on the access operation,data can respectively be read or written on lines 251 and 252.

If the appropriate data are not found at any of the blocks within theline, then other data are evicted from the cache to make room for thenew data. If the evicted data are dirty, i.e., the version of the datastored in the cache is modified and the copies of the dirty data storedin surrounding levels of the memory hierarchy are not consistent, thenthe evicted data may need to be written back to appropriate addresses inthe surrounding levels of the memory hierarchy to maintain consistency.

The goal is to sample memory system transactions in order to gainevidence about the behavior of the memory system and programs executingon the computer system. Each memory transaction is inspected as itenters the memory system to determine if this particular transactionshould be selected for sampling. Two functions control whichtransactions to sample: the trigger function 250; and the selectionfunction 260.

The trigger function 250 determines when the selection function 260should be activated, while the selection function 260 determines whichmemory transactions should be sampled, once the trigger function hasactivated the selection function. In the most general case, each ofthese functions can operate as a function of any memory system or memorytransaction state.

Selection Function

The selection function 260 is enabled via an enable line 266 that turnson the selection function when the counter 265 reaches a specifiedvalue. The maintenance of the value is described below. The selectionfunction accepts as input information about the transaction on line242″, as well as status information about the transaction on line 267.The job of the selection function is to decide if the transaction is ofinterest, and if so, to forward information about the transaction to thesampling buffer 300 via line 299.

In the general case, a monitor register (MON) 263 inside the selectionfunction logic stores state used to match against memory systemtransactions. In the particular case of monitoring accesses to aspecific cache block, the monitor register 263 can store the numbers ofone or more blocks to be monitored. The monitor register contents (suchas block numbers) can be loaded into the register by hardware orsoftware via line 261. Addresses of other regions of memory can also bestored in the monitor register.

One way the selection function 260 can be implemented is by predefininga set of different selection functions, and employing a mode register264 within the selection function. The mode register 264 can be loadedwith a mode value via line 262 to control the particular predefinedselection function to use during sampling. The various selectionfunction modes might include functions that select transactions that:

reference a particular level in the memory hierarchy;

reference a particular region of memory within a particular level of thememory hierarchy. The particular region can include one or more cacheblocks within one or more cache lines, one or more cache lines, or oneor more contiguous regions of main memory addressed by either virtual orphysical addresses;

have a particular type, e.g. read, write, or invalidate;

hit in a particular cache memory;

miss in a particular cache memory;

cause a particular cache protocol state transition e.g., dirtyevictions;

originate from a particular source, e.g., an instruction executing inthe processor pipeline, an instruction execution from a particularcontext, process, thread, or address space, direct memory access from aninput/output device, or cache coherency messages in a multiprocessorcomputer system.

Selection functions can additionally be composed using booleancombinations (AND, OR, and NOT) of these selection criteria.Alternatively, the selection function can be implemented withprogrammable logic controlled by software to provide additionalflexibility.

After the selection function has identified a memory transaction to besampled, the state information is captured and recorded in one of thesampling buffers 300-302. The state information is collected as theselected transaction is processed by the various levels of the memoryhierarchy.

Several implementation techniques are possible. For example, a “selectedtransaction” field (such as a single bit) can be associated with eachmemory transaction. The field causes logic circuits in the memory systemto record information at appropriate points during the processing of theselected transaction when the bit is set. An alternative implementationuses a comparator registers at appropriate points in the memory systemhierarchy to compare identifying information from each memorytransaction with the identifying information of a selected transaction,and if they match, record relevant state information.

Modes of Operation

Consider the implementation of the selection function using comparatorregisters to choose selected transactions at each level of the memoryhierarchy. Restricting attention to a single level of the memoryhierarchy consisting of a single cache memory, the selection functionmay specify a particular region of the cache to monitor, such as a setof cache blocks. If the index portion of the transaction informationcarried on line 242 is identical to one of the block indices stored inthe monitor register in the selection function 260, then informationabout the state of the indexed cache block is captured and recorded inone of the sampling buffers as described in detail below.

Some state information can be captured before the transaction isprocessed by the memory system, and additional state information can becaptured after the transaction completes. After a specified number oftransactions have been recorded, for example, when any of the samplingbuffers 300-302 are full, a read signal can be generated on line 271.The read signal 271 can be in the form of an interrupt, a softwarepollable value set in a register, or an exception condition.

In response to the read signal, the sampling software 280 can read thestate information stored in the sampling buffer for further processingvia line 272. It should be noted, that multiple buffers 300-302 can beused to collect multiple samples. Increasing the number of buffers canamortize the cost of sampling overhead, by transferring more than onesample per read signal.

Trigger Function

The loadable counter 265 is initialized with count-down values on line268. The counter 265 is decremented using trigger events on line 254.Trigger events can be clock cycles on line 251 or transactions on line252. Which trigger event to use can be selected on line 253.

Whether or not a trigger event on line 254 decrements the counter 265 iscontrolled by the trigger function 250. The trigger function can be anyarbitrary function of the state of a memory transaction which can bedetermined via information arriving via lines 242″′ and 267. Thefunction 250 can be implemented with two internal loadable registers asdescribed above for the selection function.

Some specific examples of useful trigger functions include those thatmatch on:

any memory transaction;

memory transactions that reference a particular level of the memoryhierarchy, e.g., a particular cache;

memory transactions that hit in a particular level of the memoryhierarchy, e.g., a particular cache;

memory transactions that miss in a particular level of the memoryhierarchy;

memory transactions that experience certain cache protocol statetransitions, e.g., dirty evictions;

memory transactions that access a particular region of memory, e.g.,range of addresses, a particular cache line, a particular cache blockwithin a particular cache line, a particular region of the cache, etc.;

memory transactions from a particular source, e.g., from the processorpipeline, from a particular direct memory access (DMA) device, coherencytraffic from another processor, etc.; and

memory transactions of a particular type, such as read transactions,write transactions, or invalidate transactions;

The use of the trigger function 250 enables the sampling hardware toskip a specified number of transactions before applying the selectionfunction to the stream of memory references. In a sophisticated example,this would allow one to count three accesses to a particular block, andthen to gather memory transaction samples for the next two misses tothat block.

In another useful example, one can trigger selection after an access toa particular cache block by a particular context (such as a process orthread), and then gather samples for a specified number of subsequenttransactions to the same block by different hardware, process, or threadcontexts.

Therefore, there are two steps to sampling:

1) determining a matching transaction, and then

2) deciding to keep or discard sampling data related to the matchingtransaction so that sampling can take place both in spatial and temporaldimensions.

The countdown register 265 can be reset via line 268. For randomsampling, the initial value written into the countdown register can bechosen randomly from an interval of numbers, and the random number canbe computed either in software or via a hardware circuit capable ofgenerating random numbers. It should be noted that the register 265 canalso count upwards.

Sampling Buffer

FIG. 3 shows the details of how one of the buffers 300-302 is allocated.The buffer can be implemented as a set of software readable registers,or other types of memories. The buffer includes a status field 310, anaddress field 320, a context field 330, an access source field 340, aninstruction field 350, a latency field 360, and fields 370 for otherstates.

The status field 310 can include block status information and cachestate protocol information such as whether the block is dirty or clean(modified or not), shared (one or more execution threads can access thedata), exclusive (non-shared), valid or invalid (the data arelegitimate), and cache hit or miss status. It can also hold informationsuch as the particular line index number and block number accessed bythe transaction in a cache memory. If there are multiple levels in thememory hierarchy, then there can be multiple copies of field 310, eachfield storing status information about the transaction for a particularlevel of the memory hierarchy.

The address field 320 can store the virtual and/or physical addresses ofthe data accessed by the transaction being sampled.

One concern for hardware implementation may be the number of wiresrequired to route the physical and virtual addresses to the buffer 300,for example, about 47 wires or so for the virtual address, plus 40 or sowires for the physical address. In computer systems which supportsoftware-managed TLBs, the number of wires can be reduced by simplystoring the index of the TLB entry that performed thevirtual-to-physical translation in the address field 320, along with theoffset into the referenced page. Then, the software 280 can read theentry from the specified TLB entry to determine both the virtual andphysical addresses.

Note this technique relies on two properties.

The first property requires that the TLB entry of interest has not beenreplaced between the time the information was recorded and the time thesoftware reads the TLB entry. In cases where the TLB implements someapproximation of a least-recently-used (LRU) replacement policy, as willbe the general case, this will not be a problem, because the entry inquestion will have been used recently by virtue of having been involvedin a recent cache access.

The second property requires that software can read the TLB entry. Incases where direct reading of the TLB is not possible, software canmaintain a shadow copy of the contents of the TLB.

The context field 330 can store the address space number (ASN), thehardware context identifier (HCI) in case of a multi-threaded processor,a process identifier (PID), and/or a thread identifier (TID) of thesource of the memory transaction when the source is an instructionexecution in the processor pipeline. The field can also store theaddress space number (ASN) referenced by the memory transaction causedby the instruction.

The source field 340 can be used to store the source of the access,e.g., a load or store instruction, a DMA request, or cache coherencyprotocol operation, as well as additional information to identify thesource.

If the source of the access was an instruction execution, then theprogram counter (PC) of the instruction that caused the access can bestored in the instruction field 350. The program counter field 350 canalso be used to store information about other kinds of sources to save aregister). For example, if the source is a coherency operation fromanother processor in a multiprocessor computer system, then the field350 can be used to hold the processor number of the processororiginating the request that caused the coherency operation. For DMAtype of transactions, the identity if the I/O device that initiated theDMA can be stored.

The time interval (latency) between successive accesses and/or theinterval from when the request was issued until the data arrives in theprocessor (or in the case of a write, the interval from when the datawas sent to the memory until the data was committed into the memory) canbe stored in the latency field 360. The interval can be measured interms of processor clock cycles, or the interval can be measured inother units such as the number of transactions processed by the memorysystem. The interval can also be broken down into the time required toprocess the transaction at each level of the memory hierarchy.

Additional registers such as field 370 can be added to this structure tostore additional memory system state that is captured at the time thatthe sampled memory transaction is processed. This state can includeinformation about memory system transactions that have occurred sincethe last sample, such as counts of the total number of transactions, orof transactions meeting a particular set of criteria.

As shown in FIG. 2, sample events on line 290 which can be part of thesampled state can include hit/miss, valid/invalid, dirty, and the like.A select signal on line 291 can determine which particular event tosample.

Other state information can also include contents or number of validentries in memory system structures such as write buffers, victimcaches, translation lookaside buffers (TLBs), miss-address files (MAFs),and memory transaction queues.

Random Memory Transaction Sampling Techniques

In a preferred embodiment, transactions which access any level of thememory hierarchy are sampled using at least two modes of operation. In afirst mode, accesses to specific regions (addresses) of memories aresampled. In a second mode, randomly selected memory transactions aresampled to estimate the performance of any part of the memory hierarchy.

In the first mode, the most interesting information can be collected bysampling at least two consecutive transactions to the same physicallocation, by way of example, a cache block. This will reveal cache statetransitions.

In the second mode, randomly sampling a large number of transactionsover time will allow a statistical analysis to estimate overall memoryperformance, without seriously impacting the throughput of the system.In other words, random sampling allows one to measure memory performancein actual operational systems.

Therefore, the apparatus and method disclosed herein can sampletransactions to: specific cache blocks clean or dirty, specific regionsof memories, all memory locations, all memory locations where cacheblocks are dirty, to memory locations where the data are not in thecache.

Because cache state transitions are of particular interest, thearrangement shown in FIG. 2 is designed to store copies of stateinformation for at least two consecutive accesses when the mode is cachesampling. The first copy captures information about the state of thecache block after the first access. The state after the second access isstored as the second copy. By comparing these two states, it is possibleto determine what transitions must have occurred in the system's cachecoherency protocol.

The notion of storing state information for successive transactions canbe generalized to sequential accesses that match simple,software-specified criteria, such as successive misses, successive hits,successive invalidations, and so forth. These are different variationson the cache sampling mode set via line 262.

In the preferred embodiment, the state and address information iscaptured when a cache block is updated, so there is no need to read thecache directly. By capturing the information on its way into the cache200, the hardware design is simpler because the need to run wires fromeach cache block to the sampler 270 is avoided. The design can also besimplified by limiting the sampling to a small number of blocks. Whenonly a small number of blocks are concurrently sampling, extra hardwarefor each cache block is avoided.

By using software to load the monitor register 264, flexible control ofa wide range of monitoring techniques to be implemented. For example, ifno access activity is detected for a particular cache block within aspecified period of time, then the software can abort the monitoring ofthat block by simply specifying another block to monitor. In anothermode, the monitored cache block can be chosen randomly, in order tostatistically sample the behavior of each block in the cache.Alternatively, software can sample the blocks in a round-robin order.

It is also possible to selectively monitor a cache block associated witha specified program variable or data structure. Here, softwaredetermines which cache block will store a particular variable, andselects that block as the one to monitor. This technique enablesprogrammers to interactively debug memory system performance byidentifying conflicts in executing programs that should be investigated.Similar techniques can be used by adaptive runtime software to avoidsevere cache conflicts through dynamic data relocation.

It should be noted, that the transaction sampling as described hereincan be employed for different levels in the memory hierarchy byduplicating the sampling hardware shown in FIG. 2. Note this techniquerelies on the two TLB properties described above.

The sampling techniques as described herein permit a fine-grainedmonitoring of memory transactions with low overhead hardware overhead.This information can be used in many ways. For example, the collectedinformation can help system designers to better understand theperformance of the memory sub-systems, such as caches, DRAM, and thelike. The performance data can be used to guide optimizations.

Estimating Memory Interactions Among Contexts

The sampled information about memory system transactions can be used tocompute a variety of statistics about memory system activity. Thisprocess 400 is shown in FIG. 4. The process involves repeatedlyselecting a region of a memory to monitor, e.g., a specific cache blockwithin a specific set-associative cache line within a specific cachememory, step 410, recording state information from multiple consecutivememory transactions that access this region step 420, and communicatingthis recorded state information to software step 430.

After a predetermined number of samples have been collected or apredetermined amount of time has elapsed, the sampling, software canstatistically analyze the recorded information to estimate a variety ofproperties about cache utilization and memory system interactions amongcontexts.

A typical way of using this approach is for the software to periodicallychoose a random cache block to monitor, and collect a sequence ofsamples for transactions that access this particular block. After agiven period of time has elapsed, the software chooses a new randomcache block to monitor, and the entire process 400 can be repeated. Overtime, samples will accrue for all blocks in the cache.

This random sampling in the spatial dimension permits the estimation ofstatistics concerning sharing and conflicts in both space and time. Notethat the random choice of the region to monitor can also be implementedby hardware logic. In general, it is also possible to monitor eachregion until a specified number of sample events 290 have occurred. Theevents 290 can be memory transactions to the region, or total memorysystem transactions, or elapsed time measured in processor clock cyclesor other units.

For each transaction, it is possible to capture a variety ofinformation, as described herein. For these analyses, the information ofinterest about each transaction includes its hit or miss status in thecache of interest, cache protocol state information about the blockreferenced by the transaction, the type of transaction (e.g., read,write, or invalidate), the virtual and physical addresses referenced bythe transaction, the corresponding location within the cache (such asthe block and/or line indices).

Additional recorded information identifies the context of thetransaction, such as a cache coherency operation from another processor,a direct memory access from an input/output device, or an instructionexecution from a particular process, thread, address space number, orhardware context.

The analysis of the samples includes first selecting a subset of thesamples that are of interest step 440. In particular, a subset may bepairs of samples from consecutive accesses to a region, such as aspecific cache block that also match additional functional criteria.

Because the selected transaction pair resulted from consecutive accessesto the same physical location in the cache, information recorded aboutthe transactions can be used to estimate sharing or conflicts for thisphysical space in the cache. Also, statistics about frequencies ofvarious state transitions in the cache protocol can be determined,because examining the protocol state on two consecutive transactionsidentifies the transition that must have taken place to go from thestate in the first sample of the pair to the state in the second sampleof the pair.

To estimate sharing, the analysis selects pairs where the secondtransaction in the pair was a cache hit in step 450. This indicates thatthere was sharing between the first and second transaction. By examiningthe context identifying information associated with both samples, it ispossible to determine which contexts usefully shared this physical spaceduring the sampled time interval. By aggregating this information overmany such pairs of samples, one can statistically estimate metricsconcerning both intra-context and inter-context sharing of physicallocations.

One useful metric is determined by counting the number of pairs wherethe first pair in the sample matches one specific context, and where thesecond pair in the sample matches a second specific context, effectivelyyielding a matrix of counts that is indexed by the identifiers of thefirst and second contexts. Similarly, by analyzing pairs where thesecond sampled transaction experienced a cache miss in step 460, one canstatistically estimate metrics concerning intra-context andinter-context conflict for physical locations.

An alternative use of this hardware is to choose a specific cache regionto monitor. The chosen region corresponds to the space in the cache thatstores a particular program variable or data structure. By collectingsamples and filtering the samples to obtain samples pairs where at leastone of the transactions involves the variable or data structure ofinterest, it is possible to estimate cache conflict rates and toidentify other specific program variables or data structures that arethe sources of the conflicts.

This estimation can be done dynamically to enable on-line programdebugging or optimization of performance problems within a runningprogram or system. This technique enables programmers to interactivelydebug memory system performance by identifying conflicts in executingprograms that are investigated. Similar techniques can be used byadaptive runtime software to avoid severe cache conflicts throughdynamic data relocation.

Estimating Statistics of Properties of Memory Transactions

The sampled information about memory system transactions can be used tocompute a variety of statistics about memory system activity. Thisprocess 500 is illustrated in FIG. 5, and is accomplished using thehardware previously described by means of the following steps:

Step 1: Choose a selection function to identify memory transactions ofinterest 510.

Step 2: Record information about selected memory system transactions520.

Step 3: Communicate the recorded state information to software 530.

Step 4: Select a subset of the transactions that are considered to be ofinterest 540.

Step 5: Analyze this subset to estimate various statistics or properties550.

The recorded state information includes a wealth of information abouteach memory transaction, so many useful statistics can be computed.

Information in the samples can include:

addresses referenced by the memory transaction;

context identifying information, such as a process identifier, threadidentifier, hardware context identifier, address space number, a directmemory access device identifier, or a processor identifier of cachecoherency traffic;

status information for each level in the memory hierarchy referenced bythe transaction, such as cache hit/miss, dirty/clean, and/orshared/exclusive status;

Analyze the Recorded State Information

Sampling individual memory system transactions makes it possible tocompute a variety of statistical metrics about distributions ofproperties of memory system behavior. For example, it is possible toestimate distributions of latencies to service memory requests, or toestimate rates of cache hits at a particular level or region in thememory hierarchy. Filtering mechanisms can be used to identify subsetsof the recorded transactions that are of interest, permitting thestatistics to focus in on particular aspects of the memory system thatare of interest, such as transactions to a particular region or level inthe memory hierarchy, or a particular class of transactions such asreads, writes, or invalidates.

After a set of samples of interest has been identified, standardstatistical techniques can be used to derive averages, standarddeviations, histograms and other statistics about the samples ofinterest. Averages can be used to estimate rates of occurrence forparticular events online 290 of FIG. 2, such as cache hits or misses, orevictions.

It is also possible to estimate the fraction of requests due to reads,writes, or invalidates. These rates can also be estimated with respectto a particular context, so as to estimate metrics such as cache hitrates per process, or average memory system latency experienced by athread. It is also possible to estimate the fraction of a level of thememory hierarchy that is being consumed by a particular context.

Standard error estimation techniques can be used to obtain confidenceintervals on the accuracy of the derived statistics. In particular, forstatistics that involve a number of samples with a specific property,error bounds can be approximated using the reciprocal of the square rootof the number of samples with that property. These error bounds can alsobe used to dynamically control the rate at which selected transactionsare sampled, so as to tradeoff accuracy with sampling overhead.

When the recorded state information includes latency information, eitherin the form of the latency required to process the memory transaction,or in terms of the latency between two consecutive sampled memorytransactions, the information can be used to compute latency-basedstatistics. Latency is typically measured in units of time, such asprocessor clock cycles, but may also be measured in other units, such asthe number of memory transactions processed.

Instruction and Data Relocation

In a very general sense, processors execute instructions that operate ondata. In many modem computer systems, the instructions and data areusually maintained as separate structures using different memory pagesbecause the access patterns for instructions is quite different thanthat for data. Virtual to physical memory mapping for instructions anddata is usually performed by the operating system. Alternatively,relocation of structures can be done manually, or by compilers, linkers,and loaders. Some systems can relocate structures dynamically as theinstructions execute.

Using the hardware described herein, it is possible to give feedback toa variety of interesting pieces of software. For example, sampled memorytransaction state information can be used to drive, for example, pageremapping policies, or to avoid self-interference by providing feedbackto compilers, linkers, or loaders.

For example, software can aggregate conflicting addresses at the pagelevel to inform dynamic page-remapping algorithms implemented byoperating systems. It is also possible to provide interesting profilingtools that identify potential performance problems to programmers andusers.

For example, it is now possible to estimate how often data are dirtywhen the data are evicted, and how often DMA transfers or cachecoherency protocol transactions occur, giving a sense of how effectivelythe memory system is being used.

Page Replication and Migration in a Multiprocessor Computer System

In non-uniform memory access (NUMA) multiprocessor systems, eachprocessor has portions of the memory system that it can access morequickly (or with higher bandwidth) than can other pieces of the memorysystem. In order to improve performance, data (which can either beprogram data or instructions) that are frequently accessed by aprocessor can be moved to a region of the memory system that can beaccessed more quickly by that processor.

This motion can be accomplished in two ways. The data can be replicatedby making multiple copies of the data. Ideally, the data are judiciously“scattered” throughout the memory system. Alternatively, the data can bemigrated by actually moving the data into a lower latency or higherbandwidth memory. The steps 600 involved are illustrated in FIG. 6 andinclude the following:

Step 1: Record information about selected memory system transactions610.

Step 2: Identify frequently accessed regions of memory (e.g., pages)620.

Step 3: Identify candidates for replication and migration 630.

Step 4: Replicate and/or migrate appropriate data to improve specificmetrics 640.

The key to this process is embodied in Step 2 and Step 3. Informationabout which pieces of data, e.g., information about referenced virtualand physical addresses, that are being frequently accessed by whichprocessor, and also which pieces of data incur substantial cache missesor are experiencing high latencies in their access can be used to guidereplication and/or migration decisions.

Information about the type of accesses (e.g., reads, writes, andinvalidates) can further guide the decision of whether to replicate orto migrate, or to leave the data in place. For example, data that arefrequently written by multiple processors (e.g., write-shared pages)should probably not be replicated or migrated, while data that arefrequently read but only infrequently written (e.g., read-shared pages)are good candidates for replication. Pages that are heavily accessed byonly a single processor are good candidates for migration to a memorythat is closer to the accessing processor. This information can begathered by statistical sampling of memory system transactioninformation as described herein.

The information about memory system transactions can be aggregateddynamically and can be used in an on-line manner to dynamically controlthe replication and migration policies of the computer system.Typically, replication and migration are handled by the operatingsystem, but they can also be handled by other software or hardwarelayers.

There are several potential performance metrics that replication ormigration policies can attempt to improve, including an increase intotal system throughput, an increase in throughput for particularhigh-priority jobs, a decrease in traffic between processors andmemories, a decrease in total memory latency, or an overall increase insystem performance.

Context Scheduling

Because caches in a hierarchical memory shared data originating fromvarious hardware contexts, threads executing in different hardwarecontexts compete for lines in a cache. Therefore, it is desired toschedule threads so that resource conflicts are minimized.

Judicious scheduling is especially important for multithreadedprocessors where memory references from different hardware contexts areinterleaved at a very fine-grained level, and the relevance is increasedwhen these contexts share memory system resources, such as caches.However, it is also important for single-threaded processors when thecaches are large enough relative to the number of memory transactionsmade by a thread during a scheduling quantum. Then there is some hope ofretaining some useful cache contents when the next quantum is allocatedto a particular context. All of these scheduling decisions candynamically adapt to feedback gathered from statistical sampling ofmemory system transactions during on-line operation.

This can be done by sampling memory system transaction information asdescribed herein. Operating system software can benefit from consideringvarious aspects of memory reference patterns of threads or processeswhen making scheduling decisions. This process 700 is illustrated inFIG. 7.

Step 710 samples transactions for specified contexts. By capturing oldand new context identifiers as part of a cache monitor, the operatingsystem software can statistically estimate the degree to which differentcontexts are sharing and conflicting in the cache in 720. Theseestimates can be used by context schedulers in steps 731-733 topreferentially schedule contexts. The scheduling decisions 731-733,described below, can benefit from considering various metrics, includingincreasing the amount of sharing among contexts competing for memoryresources or decreasing conflicts sharing among contexts competing formemory resources.

Co-scheduling

For example, it makes sense to preferentially co-schedule a thread thathas a large cache footprint concurrently with a thread that is makingonly modest use of the cache, because the memory system demands of suchthreads complement each other, thereby increasing sharing. Also, itmakes sense to use, as much as possible, non-overlapping regions of thecache.

On the other hand, the operating system software should strive tominimize resource conflicts, for example, avoiding co-scheduling twothreads with large cache footprints, where possible, because this willresult in many more conflict misses as the threads evict each othersuseful data from the cache, thereby decreasing conflicts.

Share-based Scheduling

Share-base or proportional-share scheduling policies ideally want togive each context a maximal share of each cache memory in the memoryhierarchy. With the present sampling technique, it is possible tostatistically estimate the portion of cache occupied by each context instep 720. This allows the scheduler to base its decisions on metricssuch as giving each process a maximal share of the memory systemresources, effectively partitioning memory system resources amongcontexts in proportion to their needs.

Allocation-based Scheduling

Each context that can be scheduled has associated with it allocatedresources, such the amount of cache it can use. Context which use morethan their allotted share can be slowed down or suspended. Similarlycontext that underutilize their allocated share can be favored. Whilesome contexts are suspended, others can increase their share of thecache. The suspended contexts can be allowed to continue after its cacheusage has decreased as a result of increased cache pressure from otheractive contexts. This can be distinguished from known approaches thatgenerally do not allow information to be monitored at the cache line orblock level, other than through simulation.

All of these scheduling decisions can dynamically adapt to feedbackgathered from statistical sampling of memory system transactions duringon-line operation.

The foregoing description has been directed to specific embodiments. Itwill be apparent to those skilled in the art that modifications may bemade to the described embodiments, with the attainment of all or some ofthe advantages. Therefore, it is the object of the appended claims tocover all such variations and modifications as come within the spiritand scope of the invention.

We claim:
 1. A method for estimating statistics of properties oftransactions processed by a memory sub-system of a computer system,comprising the steps of: randomly selecting memory transactionsprocessed by the memory sub-system; recording states of the system assamples while the selected transaction are processed by the memorysub-system; and statistically analyzing the recorded states from asubset of the selected transactions to estimate statistics of the memorytransactions, wherein the subset of selected transactions is chosen as afunction of the recorded states, and wherein the function selects on thebasis of addresses accessed by the selected transactions.
 2. A methodfor estimating statistics of properties of transactions processed by amemory sub-system of a computer system, comprising the steps of:randomly selecting memory transactions processed by the memorysub-system; recording states of the system as samples while the selectedtransaction are processed by the memory sub-system; and statisticallyanalyzing the recorded states from a subset of the selected transactionsto estimate statistics of the memory transactions, wherein the subset ofselected transactions is chosen as a function of the recorded states,and wherein the function selects on the basis of contexts that issuedthe selected transactions.
 3. The method of claim 2 wherein the contextsinclude process identifiers, thread identifiers, hardware contextidentifiers, address space numbers, direct memory access deviceidentifiers, processor identifiers of cache coherency traffic.
 4. Amethod for estimating statistics of properties of transactions processedby a memory sub-system of a computer system, comprising the steps of:randomly selecting memory transactions processed by the memorysub-system; recording states of the system as samples while the selectedtransaction are processed by the memory sub-system; and statisticallyanalyzing the recorded states from a subset of the selected transactionsto estimate statistics of the memory transactions and to determineaverages of properties of the selected transactions, and wherein theaverages include rates of sampling events occurring between consecutiveselected transactions.
 5. The method of claim 4 wherein the samplingevents are units of time.
 6. The method of claim 4 wherein the samplingevents are any memory transactions.
 7. A method for estimatingstatistics of properties of transactions processed by a memorysub-system of a computer system, comprising the steps of: randomlyselecting memory transactions processed by the memory sub-system;recording states of the system as samples while the selected transactionare processed by the memory sub-system; and statistically analyzing therecorded states from a subset of the selected transactions to estimatestatistics of the memory transactions and to determine averages ofproperties of the selected transactions, and wherein the averagesinclude cache hit/miss rates of a predetermined context, thepredetermined context selected from a process, thread or hardwarecontext.
 8. The method of claim 7 wherein the memory sub-system includesa plurality of hierarchical levels, and wherein the averages include anamount of memory consumed by a particular context.
 9. A method forestimating statistics of properties of transactions processed by amemory sub-system of a computer system, comprising the steps of:randomly selecting memory transactions processed by the memorysub-system; recording states of the system as samples while the selectedtransaction are processed by the memory sub-system; and statisticallyanalyzing the recorded states from a subset of the selected transactionsto estimate statistics of the memory transactions, wherein the subset ofselected transactions is chosen as a function of the recorded states,and wherein the memory sub-system includes a plurality of hierarchicallevels, wherein the function selects transactions to a specified levelin the memory hierarchy.
 10. A method for estimating statistics ofproperties of transactions processed by a memory sub-system of acomputer system, comprising the steps of: randomly selecting memorytransactions processed by the memory sub-system; recording states of thesystem as samples while the selected transaction are processed by thememory sub-system; and statistically analyzing the recorded states froma subset of the selected transactions to estimate statistics of thememory transactions, wherein the subset of selected transactions ischosen as a function of the recorded states, and wherein the functionselects transactions of a specified type of memory transaction.
 11. Themethod of claim 10 wherein the specified type is selected from readoperations, write operations, invalidate operations, direct memoryaccess operations, memory coherency operations.
 12. A method forestimating statistics of properties of transactions processed by amemory sub-system of a computer system, comprising the steps of:randomly selecting memory transactions processed by the memorysub-system; recording states of the system as samples while the selectedtransaction are processed by the memory sub-system; and statisticallyanalyzing the recorded states from a subset of the selected transactionsto estimate statistics of the memory transactions, wherein the recordedstates includes status information about cache lines referenced by theselected transactions, and wherein the recorded states further includesa cache protocol state for each referenced cache line.
 13. A method forestimating statistics of properties of transactions processed by amemory sub-system of a computer system, comprising the steps of:randomly selecting memory transactions processed by the memorysub-system; recording states of the system as samples while the selectedtransaction are processed by the memory sub-system; and statisticallyanalyzing the recorded states from a subset of the selected transactionsto estimate statistics of the memory transactions, wherein the recordedstates includes status information about cache lines referenced by theselected transactions, and wherein the recorded states further includesa dirty/clean status for each referenced cache line.
 14. A method forestimating statistics of properties of transactions processed by amemory sub-system of a computer system, comprising the steps of:randomly selecting memory transactions processed by the memorysub-system; recording states of the system as samples while the selectedtransaction are processed by the memory sub-system; and statisticallyanalyzing the recorded states from a subset of the selected transactionsto estimate statistics of the memory transactions, wherein the recordedstates includes status information about cache lines referenced by theselected transactions, and wherein the recorded states further includesa shared/non-shared status for each referenced cache line.
 15. A methodfor estimating statistics of properties of transactions processed by amemory sub-system of a computer system, comprising the steps of:randomly selecting memory transactions processed by the memorysub-system; recording states of the system as samples while the selectedtransaction are processed by the memory sub-system; and statisticallyanalyzing the recorded states from a subset of the selected transactionsto estimate statistics of the memory transactions, wherein the recordedstates includes status information about cache lines referenced by theselected transactions, and wherein the recorded states further includesa valid/invalid status for each referenced cache line.
 16. A method forestimating statistics of properties of transactions processed by amemory sub-system of a computer system, comprising the steps of:randomly selecting memory transactions processed by the memorysub-system; recording states of the system as samples while the selectedtransaction are processed by the memory sub-system; and statisticallyanalyzing the recorded states from a subset of the selected transactionsto estimate statistics of the memory transactions, wherein the memorysub-system further includes a plurality of hierarchical levels, andwherein the recorded states includes the amount of time required toprocess a particular selected transaction at each level in thehierarchy.
 17. A method for estimating statistics of properties oftransactions processed by a memory sub-system of a computer system,comprising the steps of: randomly selecting memory transactionsprocessed by the memory sub-system; recording states of the system assamples while the selected transaction are processed by the memorysub-system; and statistically analyzing the recorded states from asubset of the selected transactions to estimate statistics of the memorytransactions, wherein the recorded states further includes addressesreferenced by the selected transactions.
 18. A method for estimatingstatistics of properties of transactions processed by a memorysub-system of a computer system, comprising the steps of: randomlyselecting memory transactions processed by the memory sub-system;recording states of the system as samples while the selected transactionare processed by the memory sub-system; and statistically analyzing therecorded states from a subset of the selected transactions to estimatestatistics of the memory transactions, wherein the recorded statesfurther includes identifying context information from the context thatissued the selected transaction.
 19. The method of claim 18 wherein theidentifying context information includes a process identifier, a threadidentifier, a hardware context identifier; an address space number; adirect memory access device identifier; a processor identifier of cachecoherency traffic.